CONTROLLING DEVICE USING VISUAL CUES TO INDICATE 
APPLIANCE AND FUNCTION KEY RELATIONSHIPS 



BACKGROUND 

5 The following relates generally to controlling devices and, more particularly, to a 

controlling device that uses visual cues, such as color, to indicate appliance and function 
key relationships. 

Manufacturers typically provide a remote control with an appliance and, as such, 
different appliance types of different manufacturers are often commanded with different 

1 0 remote controls. To minimize the number of individual remote controls a user requires, 
universal remote controls have been developed. Accordingly, universal remote controls 
for commanding various functions of various types of appliances of various 
manufacturers have become quite widespread. By way of example, universal remote 
controls are described in commonly assigned U.S. Patent Nos. 4,959,810, 5,255,313 and 

15 5,552,917. 

Universal remote control applications for use in connection with hand-held 
devices such as personal digital assistants ("PDAs") are also known in the art. In this 
regard, the universal remote control applications effectively turn the host computing 
device into a universal remote control capable of being used to command various 
20 functions of various types of appliances of various manufacturers. By way of example, 
commonly assigned U.S. Patent No. 5,778,256 discloses a PDA having a separate 
infrared generating device connected to its printer port for controlling home appliances 
and commonly assigned U.S. Published Patent Application No. 2003/0103088A1 
discloses a hand-held electronic device, such as a PDA, having a remote control 
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application user interface that functions to display operational mode information to a user 
and usable, among other things, to setup the remote control application to control 
appliances for one or more users in one or more rooms, to perform activities, and to 
access favorites. 

5 Still further, NoviiRemote provides a universal remote control application for use 

in connection with a Palm brand PDA. In the graphical user interface of the 
NoviiRemote, groups of function keys are color coded according to a logical category, 
e.g., volume function control keys are always displayed using a red color, transport 
function control keys are always displayed using a yellow color, and channel function 

10 control keys are always displayed using a green color. While this allows a user a readily 
discern the various function keys on the graphical user interface, no information is 
imparted to the user as to which appliance will be controlled upon actuation of a function 
key. The need for this imparting of information will become apparent from the 
discussion that follows. 

15 For selecting which of multiple appliances a universal remote control is to 

command, a universal remote control may allow a user to place the universal remote 
control into an operational mode whereby the function keys will be used to transmit 
commands to a "primary" target appliance that has been associated with that operational 
mode. For example, a "TV" operational mode may be selected to place the universal 

20 remote control into an operational mode whereby function keys are used to transmit 

commands primarily to a designated television, a "VCR" mode may be selected to place 
the universal remote control into an operational mode whereby function keys are used to 
transmit commands primarily to a designated VCR, etc. It is to be appreciated, however, 
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that within an operational mode, certain function keys may nevertheless be "locked" so as 
to transmit commands to a target appliance that does not correspond to the "primary" 
target appliance that has been associated with that operational mode. For example, 
volume function keys may be "locked" to an amplifier such that, when the universal 
5 remote control is in a "TV" operational mode activation of a volume function key will 
cause the universal remote control to transmit a volume control command to the 
designated amplifier, as opposed to the TV. 

A universal remote control may also be provided with a "home theater" 
operational mode, a "user" operational mode (i.e., one established for a particular user), 

10 "room" operational mode (i.e., one established for a particular room), or the like 

(collectively referred to hereinafter as a "home theater" operational mode) which may be 
selected to place the universal remote control into an operational mode for commanding 
multiple, target appliances. For example, a "home theater" mode may be established 
whereby volume function keys are used to transmit volume control commands to an 

15 amplifier, channel function keys are used to transmit channel control commands to a 

cable box, picture control function keys are used to transmit picture control commands to 
a television, etc. 

While it may be useful to allow a user to command the operation of multiple 
appliances from an accessible set of function keys (whether through the use of "locked" 
20 function keys and/or "home theater" operational modes), users typically become 

frustrated as they often forget which appliance is assigned as a target for a function key at 
any given instant. Accordingly, a need exists for a means whereby a user may quickly 
discern a relationship between a function key and an intended target appliance. 
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SUMMARY 

To address this and other needs, the following discloses controlling devices that 
use visual cues, such as color, to indicate appliance and function key relationships, the 

5 visual cue being provided to allow a user to quickly discern an intended target appliance 
for commands transmitted in response to activation of a function key. A better 
appreciation of the objects, advantages, features, properties, and relationships of the 
disclosed controlling devices will be obtained from the following detailed description and 
accompanying drawings which set forth illustrative embodiments which are indicative of 

10 the various ways in which the principles described hereinafter may be employed. 

BRIEF DESCRIPTION OF THE DRAWINGS 
For use in better understanding the exemplary controlling devices reference may 
be had to the following drawings in which: 
15 Figure 1 illustrates an exemplary system in which the exemplary controlling 

devices may be utilized; 

Figure 2 illustrates a block diagram of exemplary components of the exemplary 
controlling devices; 

Figure 3 illustrates an exemplary controlling device having illuminable, hard 
20 function keys; 

Figure 4 illustrates a cross-sectional view of the hard function keys of the 
exemplary controlling device of Fig. 3; 
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Figure 5 illustrates an exemplary controlling device having a touch screen 
display; 

Figure 6 illustrates an exemplary controlling device having an EL display; and 
Figure 7 illustrates an exemplary television input selection menu display in which 
5 video source choices are color coordinated with the controlling device function keys. 

DETAILED DESCRIPTION 
For allowing a user of a controlling device 100 to discern a relationship between a 
function key and an appliance, the following describes controlling devices 100 that use 

10 visual cues, such as color, to indicate the relationship. As will become apparent, the 

function keys may be implemented as hard keys and/or soft keys so long as the function 
key is capable of having an appliance indicating visual cue associated therewith. In this 
regard, the association between the function key and the indicating color may be direct 
(e.g., by providing a color indicator directly to the function key) or indirect (e.g., by 

15 providing a color change to an element, display area, etc. of the controlling device 100 
that is linked - either visually or physically - to the function key). 

By way of example, Fig. 1 shows an exemplary system, including controllable 
appliances, such as a set top box ("STB") 104, a VCR 106, an audio amplifier/receiver 
108 and a television 102, as well as a controlling device 100a. The controlling device 

20 100a is capable of transmitting commands to the appliances, using any convenient IR, 

RF, Point-to-Point, or networked protocol, to cause the appliances to perform operational 
functions. While illustrated in the context of a STB 104 with VCR 106, audio system 
108 and television 102, it is to be understood that controllable appliances can include, but 



5 



are not limited to, televisions, VCRs, DVRs, DVD players, cable or satellite converter 
set-top boxes (STBs), amplifiers, CD players, game consoles, home lighting, drapery, 
fans, HVAC systems, thermostats, personal computers, etc. 

For use in commanding the functional operations of one or more appliances, the 

5 controlling devices 100 may include, as needed for a particular application, a processor 
300 coupled to a ROM memory 304, a key matrix 340 (e.g., hard keys, soft keys such as 
a touch sensitive surface overlaid on a liquid crystal (LCD) or electroluminescent (EL) 
display, or a combination thereof), transmission circuit(s) 310, receiver circuit(s) 308 
and/or transceiver circuit(s) (e.g., IR and/or RF), a non-volatile read/write memory 306, a 

10 means 302 to provide feedback to the user (e.g., LED, display, speaker, and/or the like), 
and key illumination means, as illustrated in Fig. 2. As will become apparent, the key 
illumination means may be in the form of separate elements, such as LEDs 320, 322, and 
324 associated with a hard key matrix, or may be integrated as part of the key matrix, for 
example in the case where the key matrix is implemented using a touch screen display. 

15 In the case where the controlling device 100 includes hard keys, an exemplary molded-in 
key 332 is shown as operative with key matrix circuit 330,331. The nature and function 
of keys 332 on the remote are described in greater detail below. 

As will be understood by those skilled in the art, the ROM memory 304 may 
include executable instructions that are intended to be executed by the processor 300 to 

20 control the operation of the remote control 100. In this manner, the processor 300 may be 
programmed to control the various electronic components within the remote control 100, 
e.g., to monitor the power supply (not shown), to cause the transmission of signals, and 
control the key illumination means 320, 322, and 324. The non-volatile read/write 
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memory 306, for example an EEPROM, battery-backed up RAM, Smart Card, memory 
stick, or the like, may be provided to store setup data and parameters as necessary. While 
the memory 304 is illustrated and described as a ROM memory, memory 304 can also be 
comprised of any type of readable media, such as ROM, RAM, SRAM, FLASH, 
5 EEPROM, or the like. Preferably, the memory 304 is non-volatile or battery-backed such 
that data is not required to be reloaded after battery changes. In addition, the memories 
304 and 306 may take the form of a chip, a hard disk, a magnetic disk, and/or an optical 
disk. 

To cause the controlling device 100 to perform an action, the controlling device 
10 100 is adapted to be responsive to events, such as a sensed user interaction with the key 
matrix 340, receipt of a transmission via receiver 308, etc. In response to an event, 
appropriate instructions within the memory 304 may be executed. For example, when a 
function command key is activated on the controlling device 100, the controlling device 
100 may retrieve a command code corresponding to the activated function command key 
15 from memory 304 and transmit the command code to an intended target appliance, e.g., 
STB 104, in a format recognizable by that appliance. It will be appreciated that the 
instructions within the memory 304 can be used not only to cause the transmission of 
command codes and/or data to the appliances, but also to perform local operations. 
While not limiting, local operations that may be performed by the controlling device 100 
20 may include displaying information/data, favorite channel setup, macro key setup, 

function key relocation, etc. A further, local operation is the ability to "lock" function 
keys across device operational modes as described in U.S. Published Patent Application 
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No. 2003/0025840. Examples of still further local operations can be found in U.S. Patent 
Nos. 5,481,256, 5,959,751, and 6,014,092. 

For creating a correspondence between a command code and a function command 
key, data may be entered into the controlling device 100 that functions to identify an 

5 intended target appliances by its type and make (and sometimes model). Such data 

allows the controlling device 100 to transmit recognizable command codes in the format 
appropriate for such identified appliances. Typically, intended target appliances are 
identified for each operational mode of the controlling device 100. By way of example, 
Fig. 3 illustrates a controlling device 100a having a "TV" operational mode, "AUX" 

10 operational mode, "VCR" operational mode, and "CBL" operation mode which are 

selectable through activation of a corresponding device mode selection key 110. Since 
methods for setting up a controlling device to command the operation of specific home 
appliances are well-known, such methods need not be described in greater detail herein. 
Nevertheless, for additional information pertaining to setup procedures, the reader may 

15 turn to U.S. Pat. Nos. 4,959,810, 5,614,906, and 6,225,938. It will also be appreciated 
that the controlling device 100 may be set up to command an appliance 102 by being 
taught the command codes needed to command such appliance as described in U.S. 
Patent No. 4,623,887. Still further, it will be understood that command codes may be 
pre-stored in the controlling device 100 or the controlling device 100 may be 

20 upgradeable, for example via use of receiver 308. 

Turning to Fig. 4, there is illustrated an exemplary mechanical construction of an 
illuminated key 220, such as would be used in connection with the hard keys of the 
controlling device 100a of Fig. 3. As is known in the art, a controlling device keypad 
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typically comprises a silicon rubber sheet 408 with molded-in key caps 332, in this case 
of translucent material at least in the area of the keycap(s) 332 which are to be 
illuminated. The underside of the keycap 332 is equipped with conductive carbon puck 
406 in the shape of a ring. Key contact areas 330 and 331, comprising conductive ink 

5 silk-screened onto the printed circuit board 402 in the form of two concentric rings, are 
positioned directly below the conductive puck 406 such that the key matrix circuit 
330,331 is completed when key 220 is depressed as illustrated in Fig. 4c. A surface 
mount type LED 320 is positioned directly below the center of the translucent keycap 332 
such that the key may be illuminated from below when an LED is enabled by the 

10 microcontroller 300. Although only a single, multi-colored LED is illustrated for clarity, 
it will be appreciated that multiple, different colored LEDs may be easily substituted. In 
the illustrated case, the multi-colored LED may comprise at least two individually- 
controllable junctions (e.g., two primary colored LEDs) contained in a single package. 
With such a device, multiple colors are possible (e.g., the two primary colors and the 

15 secondary color for the combined primary colors) depending upon which LEDs are 
enabled. 

By way of further example, Figs. 5a - 5b illustrate a controlling device 100b 
having a display 1 12. In this illustrated example, the display 1 12 comprises a touch 
screen that allows a user to interact with the controlling device 100b to, for example, 
20 setup the controlling device, select operational modes of the controlling device 100b, etc. 
In this regard, Fig. 5a shows a selection wheel by which operational mode selection may 
be accomplished. For example, selecting a "device" icon 110 may place the controlling 
device 100b in a device operational mode (e.g., TV, VCR, etc.), selecting a "room" icon 
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115 may place the controlling device 100b in a mode for commanding the operation of 
appliances in a designated room (e.g., living room, bedroom, etc.), or selecting a "home 
theater" icon 116 may place the controlling device 100b in a mode for commanding the 
operation of multiple appliances within a home theater system. Fig. 5b illustrates a 

5 display of example function control keys 1 1 7 in a "home theater" page. Such a display- 
centric controlling device is particularly described and illustrated in commonly owned 
U.S. Application Serial Nos. 60/264,767, 09/905,423, 09/905,432, and 09/905,396. 

By way of still further example, Fig. 6 illustrates a controlling device 100c which 
includes an electro-luminescent display 128. Unlike the controlling device 100a 

10 illustrated in Fig. 3, which includes silicon rubber keypads protruding through cut-outs in 
a hard plastic upper housing, the controlling device 100c uses a flexible, segmented 
electroluminescent ("EL") panel that is overlaid over a dome switch style key matrix. An 
example controlling device having such an EL panel is particularly illustrated and 
described in commonly owned U.S. Application Serial No. 10/410,103. Of particular 

1 5 note, the EL panel may be constructed to allow various parts of the display to be 

independently illuminated using different colors under control of the microprocessor 300 
and an EL display interface. 

The ability to independently illuminate various parts of the controlling device 
100, e.g., the keys and/or display, with different colors may be used to advantageously 

20 provide a user of the controlling device 100 with a visual indication as to which appliance 
a command should be transmitted to (e.g., the intended target) when a function key is 
activated. For example, a color association between a function key and an appliance may 
be used to provide a user with information indicative of which function keys are locked 
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to which appliances across the various "device" modes. Still further, the color 
association between a function key and an appliance may be used to provide a user with 
information indicative of which function key is assigned to which appliance when 
multiple appliances are controllable from a user interface, e.g., when the controlling 

5 device 100 is in a "home theater" operational mode. 

As discussed previously, a color indicative of an intended target appliance may be 
associated with a function key by, for example, illuminating one or more appropriately 
colored LEDs that are associated with the function key (e.g., in the case of "hard" keys), 
by displaying an icon, area surrounding an icon, etc. with the appropriate color (e.g., in 

10 the case of "soft" keys), or by illuminating one or more appropriately colored segments 
of an EL panel that are associated with the function key (e.g., in the case of "EL display" 
keys). In any of these instances, a color that is selected so as to be indicative of an 
intended target appliance may be predefined or user-selectable. Furthermore, the absence 
of a color being associated with a function key may also be indicative of an intended 

15 target appliance. 

By way of example, Fig. 3 illustrates a controlling device 100a having hard keys 
in which the volume control function keys 130 (e.g., volume up 130a, volume down 
130b, and mute 130c function keys) have as their intended target the appliance designated 
as the intended target when the appliance is in the "AUX" device mode. The relationship 

20 between the function keys and the "AUX" device as the intended target may be presented 
to the user by causing each of the volume control function keys 130 to be illuminated, by 
means of illuminating the appropriate LED(s), with a red color. In this case, the user may 
simply know that the color red is indicative of a function key having the "AUX" device 
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as its intended target. The relationship between the function keys and the "AUX" device 
as the intended target may also be presented to the user by causing the "AUX" device 
mode selection key 1 10a to be optionally illuminated the same color as the function keys 
that have the "AUX" device as the intended target, e.g., by also causing the "AUX" 
device mode selection key 1 10a to be illuminated red. Similarly, a relationship between 
the transport control function keys 134 (e.g., the rewind 134a, play 134b, fast forward 
134c, record 134d, stop 134e, and pause 134f function keys) and the "VCR" device as the 
intended target may be presented to the user by causing each of the transport control 
function keys 134 to be illuminated with a green color. Again, the "VCR" device mode 
selection key lOOe may be illuminated with the same green color to allow a user to 
discern the relationship between the device mode intended target appliance and the 
function keys 134. 

By way of still further example, Fig. 5b illustrates a "home theater" page of the 
controlling device 100b in which the volume control function keys 130 (e.g., volume up 
130a, volume down 130b, and mute 130c function keys) have as their intended target the 
appliance designated as the intended target when the appliance is in the "TV" device 
mode. The relationship between the function keys and the "TV" device as the intended 
target may be presented to the user by causing each of the volume control function keys 
130 (or areas surrounding or in the vicinity of the volume control function keys 130) to 
be illuminated, by means of displaying, a red color. In this case, the user may simply 
know that the color red is indicative of a function key having the "TV" device as its 
intended target. The relationship between the function keys and the "TV" device as the 
intended target may also be presented to the user by causing the "TV" device mode 
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selection key 1 10b to be optionally illuminated the same color as the function keys that 
have the "TV" device as the intended target, e.g., by also causing the "TV" device mode 
selection key 1 10b of the device selection page, illustrated in Fig. 5 a, or appropriate 
area(s) adjacent thereto, to be illuminated red. Similarly, a relationship between the 
5 transport control function keys 134 (e.g., the rewind 134a, play 134b, fast forward 134c, 
stop 134e, and pause 134f function keys) and the "DVD" device as the intended target 
may be presented to the user by causing each of the transport control function keys 134 
(or areas surrounding or in the vicinity of the transport control function keys 134) to be 
illuminated with a green color. Again, the "DVD" device mode selection key lOOf may 

10 be illuminated with the same green color, for example in the mode selection page of Fig. 
5a, to allow a user to discern the relationship between the device mode intended target 
appliance and the function keys 134. 

By way of yet another example, Fig. 6 illustrates a controlling device 100c having 
an EL display in which the volume control function keys 130 (e.g., volume up 130a, 

15 volume down 130b, and mute 130c function keys) have as their intended target the 

appliance designated as the intended target when the appliance is in the "AMPLIFIER" 
device mode. The relationship between the function keys and the "AMPLIFIER" device 
as the intended target may be presented to the user by causing each of the volume control 
function keys 130 (or areas surrounding or in the vicinity of the volume control function 

20 keys 130) to be illuminated, by means of illuminating the appropriate EL segment(s), 

with a red color. In this case, the user may simply know that the color red is indicative of 
a function key having the "AMPLIFIER" device as its intended target. The relationship 
between the function keys and the "AMPLIFIER" device as the intended target may also 
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be presented to the user by causing the "AMPLIFIER" device mode selection key 1 10c to 
be optionally illuminated the same color as the function keys that have the 
"AMPLIFIER" device as the intended target, e.g., by also causing the "AMPLIFIER" 
device mode selection key 1 10c, or appropriate area(s) adjacent thereto, to be illuminated 

5 red. Similarly, a relationship between the transport control function keys 134 (e.g., the 
rewind 134a, play 134b, fast forward 134c, stop 134e, and pause 134f function keys) and 
the "DVD" device as the intended target may be presented to the user by causing each of 
the transport control function keys 134 (or areas surrounding or in the vicinity of the 
transport control function keys 134) to be illuminated with a green color. Again, the 

10 "DVD" device mode selection key lOOf may be illuminated with the same green color to 
allow a user to discern the relationship between the device mode intended target 
appliance and the function keys 134. 

If, for example, the volume transport function keys 134 and the volume control 
function keys 132 are locked to their respective devices, the transport function keys 134 

15 and the volume control function keys 132 may continue to be illuminated using an 

appropriate device-indicative color even when the controlling device 100 is placed into 
others of its various home theater, room, or device operational modes. It will be 
appreciated that the LEDs, EL segments, display, etc. need not be constantly illuminated 
in a device-indicative color but that a key or the like may be activatable by the user to 

20 temporarily illuminate the LEDs, EL segments, display, etc. in an appropriate color given 
the present operational mode of the controlling device 100 and the intended target 
appliance(s) of the function keys within that operational mode. 
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While the foregoing describes controlling devices 100 that use color to indicate 
function key and appliance relationships, it is contemplated that other identification 
schemes (which may used in addition to or in lieu of color) may be provided to indicate 
function key and appliance relationships. For example, function key and appliance 
relationships may be indicated by controlling the tint, contrast, or brightness of displayed 
function keys and/or area(s) visually associated with function keys. Still further, function 
key and appliance relationships may be indicated by providing a visually identifiable 
pattern, shape, icon, or alphanumeric tag to a function keys and/or area(s) visually 
associated with function keys (e.g., imposing a Crosshatch or other pattern on all function 
keys associated with a particular appliance, using commonly shaped function keys for an 
associated appliance, surrounding each function key with (or appending in 
super/subscript fashion) a shape or icon representing the associated device, appending an 
alphanumeric tag on or near a function key indicating the associated appliance, etc.). Yet 
further, function key and appliance relationships may be indicated by associating a 
distinct audible sound, such as sounds that repeat in different numbers, are emitted using 
different frequencies, and/or are emitted in different sequences, to a function key that is 
representative of an intended target appliance. In this manner, activation of a function 
key may cause the controlling device to emit the audible sound that has been associated 
with the function key where the emitted sound is representative of the intended target 
appliance. 

Still further, with reference to Figs. 1 and 7 visual and/or audio cues may be 
utilized in connection with appliance setup processes. For example, in some cases an 
appliance (e.g., television 102) may be setup to receive input(s) from one or more 
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appliances (e.g., audio-visual input from cable box 104 and/or VCR 106). In such a case, 
in connection with the receiving appliance being setup to designate input or transmitting 
appliance(s), cues may be presented to the user that function to represent the transmitting 
appliance(s) (e.g., a Cable source may be provided with a yellow cue, a VCR source may 

5 be provided with a green cue, a DVD source provided with a blue cue, etc. as illustrated 
in Fig. 7). The cues provided to represent the transmitting appliances during setup 
processes may be pre-selected and/or user selectable. 

The cues utilized to represent the transmitting appliances during an appliance 
setup process may then be used to represent an intended target appliance for function 

10 keys (and device modes) of the controlling device 100. For example, in the example 
illustrated in Fig. 7, the VCR is provided with a green colored cue which is utilized to 
represent the VCR during the appliance setup process. Accordingly, the same green 
colored cue may be associated with function keys of the controlling device 100 that have 
the VCR as an intended target appliance. In certain cases, such as when the controlling 

15 device 100 is supplied with the appliance that is to be setup, it may be preferred to have 
the cue and appliance relationships predefined in both the controlling device 100 and the 
appliance to be setup to ensure consistency between the utilized cues. In certain other 
case, cue/appliance relationships utilized during the setup process can be communicated 
from the appliance being setup to the controlling device 100 (for example using IR or RF 

20 transmissions). Such a communication may include data that functions to identify the 
appliance type of any selected inputs and the cues associated with those appliance(s) 
whereby the controlling device 100 may use the data to set itself up to use the same cue 
to represent the same appliance(s) (i.e., in association with function keys when the 
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appliance is to be an intended target appliance and/or in association with a device mode 
key to which the appliance has been assigned). 

While various concepts have been described in detail, it will be appreciated by 
those skilled in the art that various modifications and alternatives to those concepts could 
5 be developed in light of the overall teachings of the disclosure. For example, it should be 
appreciated that particularly where controlling devices having multi-color display screens 
are involved (i.e., PDA or other LCD based controlling devices), many combinations and 
variations of the above described function key association features are possible without 
departing from the spirit and scope of the present invention. As such, the particular 
10 concepts disclosed are meant to be illustrative only and not limiting as to the scope of the 
invention which is to be given the full breadth of the appended claims and any 
equivalents thereof. 

All documents cited within this application for patent are hereby incorporated by 
reference in their entirety. 
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